Coding by motion compensation offers considerable possibilities of bit rate reduction in moving images, which exhibit high temporal coherence.
The acquisition of video image sequences is still today largely carried out in analogue form so that the images, once acquired and possibly transmitted and then stored in analogue formats, exhibit an appreciable share of noise in their content. Once digitised, these images are also often subjected to storage/editing operations which, in their turn, introduce noise, this time of a digital nature. Finally, an image sequence generally undergoes a succession of transformations, the result of which is characterized by spatio-temporal noise of a highly random nature. The effect of this noise on the coding processes based on motion compensation is highly penalizing in terms of coding efficiency in so far as the random nature of the noise renders any motion compensation ineffective.
The noise therefore represents a sizeable share of the bit rate of the binary stream after coding. This component exhibiting no useful information, it is generally desirable to attenuate it by filtering processes whose role is to eliminate the random spatio-temporal components in the image sequence.
Among the noise reduction techniques, one of the most appropriate techniques consists in recursively averaging the images of the sequence. For the static shots/objects of a sequence, the averaging makes it possible to attenuate the noise-specific random component while preserving, with a certain degree of faithfulness, the original shots/objects. For the moving shots/objects, the problem is trickier in so far as a temporal averaging introduces an unacceptable remanence effect (or trail effect). It is then necessary to choose between two strategies: leave the moving shots/objects intact (hence leave the noise) or else use motion compensation so as to “follow” the shots/objects in their motion and filter in the direction of the motion. The first strategy makes it possible to achieve motion-adapted (that is to say, adapted to the presence of motion) recursive filtering, the second strategy, intuitively more powerful, carries out motion-compensated recursive filtering.
To decrease the effects of noise in systems for coding images by motion compensation, known solutions consist in placing a filtering module upstream of the coding process. However, this solution constitutes a significant extra cost for the system.
In order to reduce the costs of such a system, one solution consists in incorporating the recursive filtering and coding processes rather than in cascading them.
In this approach, the coder-specific motion compensating architecture is exploited in order to carry out, in a single pass, motion-compensated recursive filtering. This leads to an architecture whose performance/set-up cost ratio exceeds, for certain coding configurations, that of the traditional solutions.
The principle of noise reduction by recursive filter relies on the absence of spatio-temporal coherence of the noise with respect to the useful signal. Therefore it is directed only at noise of a random nature such as that encountered in certain analogue transmission/storage media and, to a lesser extent that engendered by certain digital coding processes such as quantization. The information contained in the useful signal being characterized by its level of spatio-temporal coherence (superposition of moving shots/objects), the removal of noise is achieved by temporal averaging of the shots/objects contained in the sequence. To carry out this averaging, it is necessary to detect the various shots/objects of the sequence so as to characterize the nature of the motion over these shots/objects.
Various modes of implementation of the motion estimation function make it possible to achieve prediction. Certain modes use a “pixel-wise” motion estimation function, this signifying that a new displacement vector is calculated for each pixel. Other modes of implementation utilize the “block-wise” motion estimation solutions, this signifying that the same motion vector is used for the set of pixels which constitute the block (a block consisting of a matrix of contiguous pixels in the image). It is obvious that the modes based on “block wise” motion compensation may potentially exhibit limitations in performance when there is sizeable deformation of the shots/objects in motion. In this case, this results in a sizeable prediction error, which in turn gives rise to an attenuation of the effect of the recursive filter.
U.S. Pat. No. 6,122,314 discloses the principle of recursive filtering which consists in weighting the error signal of a coder employing motion compensation and differential coding.
However, this patent describes a process whose performance is not maximal in particular when the error signal is composed essentially of a high-energy noise component.